Automatic Building Extraction for 3d Terrain Reconstruction Using Interpretation Techniques

Conventional image matching techniques for DTM determination are unable overcome the disparity discontinuities in the stereo model caused by man-made structures and only supply a Digital Surface Model (DSM). In order to produce a DTM of the bare earth, the characteristics of the terrain cover, such as buildings and trees must to be determined to reduce the elevations derived from image matching to the terrain surface. An automatic approach and strategy for extracting building information from aerial images using combined image analysis and interpretation techniques is described in this paper. The approach is undertaken in three parts. In the first part, a dense DSM is obtained by stereo image matching. Multi-band classification, DSM, texture segmentation and Normalised Difference Vegetation Index (NDVI) are used to reveal building interest areas. In part two, based on the approximate building areas derived in part one, a shape modelling algorithm based on the level set formulation of curve and surface motion has been used to precisely delineate the building boundaries. Since the complex urban scene can result in wrongly extracting building regions, a method is required to evaluate the reliability of the evidence of buildings and delete those buildings that have been wrongly identified. Data fusion in remote sensing, such as the Dempster-Shafer approach, is undertaken to combine several image data sets for the purpose of information extraction. It can be used to interpret simultaneously knowledge from several data sources of the same region to find the intersection of propositions on extracted information derived from these datasets, together with their associated probabilities. In the third part, the Dempster-Shafer data fusion technique provides the theoretical basis for evaluating the reliability of the extracted buildings from the combination of the different data sources by a statistically-based classification. A number of test areas, which include buildings with different sizes, shape and roof colour have been investigated. The tests are encouraging and demonstrate that the three parts of the system are important procedures for effective building extraction, and the determination of more accurate elevations of the terrain surface. The approach used in this research for 3D reconstruction from stereo images over trees or built-up areas is based on an attempt to understand and interpret the image content, and is significantly different from the current approaches for determining elevations on the terrain. In order to provide an accurate DTM of the terrain surface, the characteristics of the terrain cover, such as buildings and trees must to be determined. INTRODUCTION One of the most challenging problems in the fields of computer vision and digital photogrammetry is 3D reconstruction of the terrain surface from complex aerial images in urban or suburban areas where buildings, roads, trees and vegetation are intermingled in an intricate and complex fashion. Digital Terrain Models (DTM) produced by stereo image matching algorithms have been one of the primary goals of cartography for many years. Recently, interest in the area has been stimulated by the need for digital orthophotos, 3D city models, 3D building reconstruction, production and management of 3D databases for urban and town planning and Geographic Information Systems (GIS) modelling. Although many automatic building extraction algorithms have been proposed by researchers (Collin et al 1998, Hanson et al 2001 and Henricsson 1998), there are no operational algorithms because each method is focused on a particular application area and is not transferable to other features. An important aspect of the proposed automatic building extraction system is that it aims to use image interpretation as well as elevation information to extract the building areas by integrating image analysis and interpretation techniques. Stereo image matching determines corresponding pixels or features in two overlapping images and is the fundamental to digital photogrammetry for elevation determination. Conventional image matching techniques only supply a Digital Surface Model (DSM). This means that matching occurs on the top of man-made objects such as buildings, or on the top of the vegetation rather than the terrain surface and hence does not represent the terrain surface [Baltsavias et al. 1995, Henricsson et al. 1997 and Tönjes 1996]. Section 2 in this paper introduces the proposed system for automatic building extraction. Section 3 and Section 4 describe multispectral image analysis and the level set shape modelling process respectively. Dempster-Shafer data fusion theory is presented in Section 5. Section 6 gives experimental results, and conclusions are drawn in Section 7. GENERAL DESCRIPTION OF AUTOMATIC EXTRACTION OF BUILDING SYSTEM PROCESSING OF STEREO AND MULTISPECTRAL IMAGES Figure 1 illustrates the architecture of the automatic building extraction system. Only the level set and Dempster-Shafter algorithms will be described in detail in this paper. The goal of these component of the process is to extract accurate building boundaries for reconstruction of terrain elevations from overlapping aerial or satellite images over a variety of terrain types and ground cover. Stereo Image Matching For DSM Since this paper concentrates on the process of recognizing building in images, the DSM obtained from LH Systems’ Socet Set v4.2 has been directly used in the subsequent stages of the system in Figure 1. A dense sample of points in the DSM is obtained in order to avoid missing some structures. The derived DSM is then interpolated to the size of the original image for further processing. The overall system consists of three main parts. Part 1 performs the matching of the stereo image pair, derives a disparity map, and produces a digital surface model (DSM). Then an analysis of the multispectral image supplies the results of multi-band classification, segmentation by classification and Normalised Difference Vegetation Index (NDVI). The four information layers shown as green in Figure 1 finally produce building interest areas. Part 2 uses information from part 1 to define an initial curve leading to the level set formulation of curve and surface motion to define the desired building boundaries, driven by an imagedependent speed function. Part 3 presents DempsterShafer fusion theory, which is used to combine different data sources to extract the correct building areas. Figure 1 Architecture of the building extraction system 2 1 3 Segmentation of classification Final building areas Dempster-Shaft data fusion theory Shape modelling using Level Set Method Extracted building areas NDVI DSM Area-based image matching Delineation of building areas K-Means unsupervised classification Multispectral images Image preprocessing Stereo image pairs Multispectral Image Analysis Multispectral image classification is typically used to detect individual object primitives. It aids in reducing the complexity of the image content for the next processing step of feature detection. In order to find building areas, K-Means unsupervised classification is used to classify the image because it is a fully automatic process. then using a post classification procedure, a segmented image can be created from a classified image. Segmentation partitions a classified image into meaningful regions of connected pixels that are contained in the same class. The NDVI (Vegetation Index) can then be used to transform the multispectral data into a single image band representing vegetation. The NDVI (Normalized Difference Vegetation Index) values indicate the amount of green vegetation present in the pixel. Extraction Building Interest Areas Now, we need to produce an equation for the evolving function which contains the embedded motion of as the level set { }. ) , ( t x ) (t 0 While multispectral images supply abundant information for land cover classification, the NDVI and DSM are two key parameters which define the difference between vegetated and non-vegetated objects. Simplistically, the areas which have heights above some limit, are likely to be either trees or buildings. Areas with low NDVI, and are above the general terrain surface are likely to be buildings, whereas areas with high NDVI and are above that surface are likely to be trees. Areas with high NDVI, with heights similar to the terrain surface are likely to be grassland or cultivated areas. Four information layers of the land cover classification, the results of the segmentation by K-means, DSM and NDVI, are input to ArcView Map Query operation to extract building interest areas. Using the chain rule in Sethian (1999, 1995), the evolution equation for can be a type of HamiltonJacobi equation. Based on the advantages of the Hamilton-Jacobi equation, in two space dimension, a numerical approximation for the evolving function can be obtained. Using the forward and backward difference approximations in , the evolving function can be described as Equation (3) and n defines iterations. 2 2 0 1 )) 0 , (min( )) 0 , ((max( ij x ij x n ij n ij D D t F ) )) 0 , (min( )) 0 , (max( 1 2 / 1 2 2 tF D D ij y ij y SHAPE MODELLING AND IMAGE SEGMENTATION WITH LEVEL SET METHOD where computes the new values at j using informa ion at j and j+1; x D The level set method for curve propagating interfaces was introduced by Osher and Sethian (1988, 1999). It is based on mathematical and numerical work of curve and surface motion by Sethian (1985), and offers a highly robust and accurate method for tracking interfaces moving under complex motions. D ij j i ij x ) 1 ( similarly computes the new values at j using info mation at j and j-1; x D computes the new values at i using infor ation at i and i+1; y D Consider a closed curve moving in a plane. Let be a smooth, closed initial curve in Euclidean plane ) 0 ( 2 R , and let be the one-parameter family of curves generated by moving along its normal vector field with speed , K is a given scalar function of the curvature. Let x be the position vector which parameterises by s, . ) (t ) (K ) , t s 0 ) 0 (